An Essay about Markov

Unless one is clairvoyant, the only temporally evolving processes which are tractable are those whose future behavior can be predicted on the basis of data which is available at the time when the prediction is being made. Of course, in general, the behavior of even such an evolution will be impossible to predict. For example, if, in order to make a prediction, one has to know the detailed history of everything that has happened during the entire history of the entire universe, ones chance of making a prediction may be a practical, if not a theoretical, impossibility. For this reason, one tries to study evolutions mathematically with models in which most of the distant past can be ignored when making predictions about the future. In fact, many mathematical models of evolutions have the property that, for the purpose of predicting the future, the past becomes irrelevant as soon as one knows the present, in which case the evolution is said to be a Markov process, the topic at hand. The components of a Markov process are its state space S and its transition rule T . Mathematically, S is just some non-empty set, which in applications will encode all the possible states in which the evolving system can find itself, and T : S −→ S is a function from S into itself which gives the transition rule. More precisely, if now the system is in state x, it will be next in state T (x), from which it will go to T (x) = T (T (x)), etc. To give a sense of the sort of reasoning required to construct a Markov process, consider a (classical) physical particle whose motion is governed by Newton’s equation ~ F = m~a (“force equals mass times acceleration”). At least in theory, Newton’s equation says that, assuming one knows the mass of the particle and the force field ~ F which acts on it, one can predict where the particle will be in the future as soon as one knows what its position and velocity are now. On the other hand, knowing only its present position is not sufficient by itself. Thus, even though one may care about nothing but its position, in order to produce a Markov process for a particle evolving according to Newton’s equation, it is necessary to adopt the attitude that the state of the particle consists of its position and velocity, not just its position alone. Of course, in that velocity is the derivative of position, the two are so inextricably intertwined that one might be tempted to concentrate on position on the grounds that one will be able to compute the velocity whenever necessary. However, this